Skip to content

MNT Part 1: feat(types): add QuarkChain MNT account and token types - #24

Draft
ping-ke wants to merge 15 commits into
goshard/basefrom
feature/mnt-core-types
Draft

MNT Part 1: feat(types): add QuarkChain MNT account and token types#24
ping-ke wants to merge 15 commits into
goshard/basefrom
feature/mnt-core-types

Conversation

@ping-ke

@ping-ke ping-ke commented Jul 3, 2026

Copy link
Copy Markdown

Summary

  • Add MNT (Multi-Native Token) support to core/types, including a QKC 6-element RLP-encoded StateAccount and a TokenBalances type
  • Remove the codegen file gen_account_rlp.go and replace it with hand-written EncodeRLP/DecodeRLP to support the QKC custom 6-field RLP format
  • Keep this PR limited to the account types and codec layer; runtime state integration and fixture updates are handled by follow-up PRs

Background

The standard Ethereum StateAccount uses 4-element RLP (Nonce, Balance, Root, CodeHash). The QuarkChain protocol extends this with two additional fields:

  • FullShardKey uint32: the shard key of the account
  • MntBalances TokenBalances: a map of non-default native token balances held by the account

This change produces a different state root from standard go-ethereum, requiring the genesis hash constants to be updated accordingly.

Changed Files

File Description
core/types/gen_account_rlp.go Deleted and replaced by hand-written RLP encoding
core/types/state_account.go Add FullShardKey and MntBalances fields and extend SlimAccount
core/types/state_account_qkc.go QKC 6-element RLP encode/decode implementation
core/types/state_account_qkc_test.go Roundtrip tests and pyquarkchain compatibility tests

PRs and Scope

PR / Branch Scope
This PR (feature/mnt-core-types) Add the QKC-aware account representation, TokenBalances, 6-element RLP codec, and extended SlimAccount in core/types. Snap sync wire-format support remains out of scope.
feature/mnt-state Integrate MNT accounts into core/state, including state objects, journaling, revert/history handling, StateDB, and snapshot reader support. flatReader.Account preserves FullShardKey and decodes MntBal into MntBalances.
feature/mnt-state-test-fixtures Update state, snapshot, pathdb, and related test fixtures and goldens for the QKC account encoding and resulting state roots.

After review, these three stacked changes will be combined into a single PR and merged.

Key Design Decisions

DefaultTokenID = 35760: Equal to TokenIDEncode("QKC"), consistent with pyquarkchain. The QKC token balance is stored in the standard Balance field, not in the MntBalances map, to preserve EIP-20 compatibility.

Test Plan

  • go test ./core/types — all pass, including pyquarkchain compatibility vector tests
  • go build ./... — build successful

Additional state integration and fixture/golden coverage is provided by the follow-up branches listed above.

Comment thread core/types/state_account.go Outdated
Comment thread core/types/state_account.go Outdated
Comment thread core/types/gen_account_rlp.go
@qzhodl

qzhodl commented Jul 8, 2026

Copy link
Copy Markdown

General note: the three inline issues above were all found in a first-pass Codex 5.5 Extra High review with no additional prompt engineering beyond asking it to review this PR. Before requesting review next time, please run a self-review pass with Codex 5.5 Extra High (or an equivalent high-reasoning review mode) and address the obvious state/snapshot/generation issues it finds first.

Comment thread params/config.go Outdated
ping-ke and others added 4 commits August 5, 2026 18:13
- Add TokenBalances type with sorted list encoding compatible with pyquarkchain
- Add StateAccount.MntBalances field and QKC 6-element RLP codec
  (replaces generated gen_account_rlp.go with hand-written EncodeRLP/DecodeRLP)
- Add uint32 RLP encoding helpers for token IDs
- Update genesis hashes to reflect QKC 6-element account encoding
- Add comprehensive tests: roundtrip, pyquarkchain encode/decode compatibility
Comment 1: SlimAccount only held {Nonce,Balance,Root,CodeHash}, so the
slim-RLP path used by SlimAccountRLP (stateupdate.go account updates/
origins), FullAccount (pathdb rollback in triedb/pathdb/execute.go) and
flatReader.Account (snapshot flat read) silently dropped MntBalances and
FullShardKey. A QKC account served from any of those paths came back with
MntBalances=nil / FullShardKey=0 and re-committed a corrupted account,
forking the trie root.

Extend SlimAccount with:
  - MntBal       []byte (rlp optional) = TokenBalances.SerializeToBytes()
  - FullShardKey uint32 (rlp optional)

MntBal uses the []byte serialization (TokenBalances holds an unexported
map, not RLP-struct-encodable) and preserves the nil-vs-empty distinction
so the 0x80 / 0x8200c0 trie encoding stays byte-stable across the slim
round-trip. Unlike the trie qkcAccountRLP.TokenBal, the QKC default
balance is NOT merged into MntBal — slim keeps it in the Balance field.
Both fields are rlp optional so pre-MNT snapshots still decode. Because
FullAccount now reconstructs both fields, the pathdb rollback path is
covered without further changes.

Extend TestSlimRLPRoundTripEquivalence with fullShardKey / MNT-only /
MNT+QKC+shard cases (direct-QKC-encode == via-slim-encode).

Comment 3: remove the rlpgen go:generate directive. StateAccount now uses
the hand-written QKC codec (EncodeRLP/DecodeRLP in state_account_qkc.go);
regenerating gen_account_rlp.go would reintroduce a conflicting standard
4-field codec that drops MntBalances / FullShardKey. Replaced the
directive with a NOTE explaining why it must stay removed.

Comment 2 (empty() must consider MntBalances) is resolved downstream on
feature/mnt-state (stateObject.empty() checks IsBlankMnt(), covered by
TestEmptyAccountWithMntNotPruned); core/state is not part of this branch.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Fixes found while reviewing the MNT account encoding:

- EncodeRLP no longer dereferences a nil *TokenBalances. When Balance is
  nil (or zero) and MntBalances is non-nil but empty, mergeQKCTokenBalances
  returns nil, and the old three-branch switch fell through to calling
  SerializeToBytes on that nil receiver. Collapsing the switch into a single
  nil-guarded path removes the whole class of gap.

- DecodeRLP rejects a non-empty optional field instead of silently dropping
  it. pyquarkchain's _Account always writes b"" there, so a non-empty value
  could only come from a foreign encoder, and discarding it would change the
  bytes on re-encode.

- Dropped the dead rlp:"optional" tag on StateAccount.MntBalances. The type
  has a hand-written codec so the tag never applied, and as written it was
  invalid (an optional field followed by the non-optional FullShardKey), which
  would break any future codec built for this struct.

- Removed qkc/common/uint32_rlp.go: qkc/common/special_rlp.go now provides
  Uint32 after it moved down from qkc/types.

Deliberately unchanged: the zero-valued-token-balance encoding is
non-idempotent (first encode 0x00c0, re-encode empty) because pyquarkchain's
TokenBalances.serialize tests len(_balances) before filtering zero balances.
Canonicalizing it would fork the account trie root. Pinned by
TestStateAccountEmptyBalancesPythonGolden.

SlimAccountRLP keeps panicking on a serialization error, matching the
surrounding geth convention.

Adds TestStateAccountEncodeBalanceMntCombinations, covering Balance
(nil/zero/non-zero) against MntBalances (nil/empty/zero-valued/non-zero) and
pinning the wire TokenBal for each.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@ping-ke
ping-ke force-pushed the feature/mnt-core-types branch from ceef3d7 to 7ef2bab Compare August 5, 2026 10:19
@ping-ke ping-ke changed the title feat(types): add QuarkChain MNT account and token types MNT Part 1: feat(types): add QuarkChain MNT account and token types Aug 10, 2026
@ping-ke

ping-ke commented Aug 13, 2026

Copy link
Copy Markdown
Author

Account RLP Compatibility with pyquarkchain

The following tables compare balance-related bytes only. The values were obtained by executing the current Go account workflow and cross-checking the resulting token-balance payloads with pyquarkchain's TokenBalances.serialize().

  • TokenBal: the payload stored in the main six-field account RLP.
  • SlimAccountRLP: the internal snapshot representation, shown as Balance and MntBal.
  • pyquarkchain: the payload produced by TokenBalances.serialize().
  • empty: empty bytes, encoded as 0x80 by the enclosing RLP field.
  • 00c0: a serialized token-balance list containing no non-zero token pairs.

The examples use:

  • QKC token ID: 35760 (0x8bb0)
  • MNT token ID: 100 (0x64)
  • Non-zero balance: 1000 (0x03e8)

New Account

A new account can be initialized with different QKC and MNT balances.

Initial values In-memory state EncodeRLP TokenBal SlimAccountRLP pyquarkchain TokenBal Aligned
QKC=0, MNT absent Balance=0, MntBalances=nil empty Balance=0, MntBal=nil empty Yes
QKC=0, MNT=1000 Balance=0, MntBalances={100:1000} 00c5c4648203e8 Balance=0, MntBal=00c5c4648203e8 00c5c4648203e8 Yes
QKC=1000, MNT absent Balance=1000, MntBalances=nil 00c7c6828bb08203e8 Balance=1000, MntBal=nil 00c7c6828bb08203e8 Yes
QKC=1000, MNT=1000 Balance=1000, MntBalances={100:1000} 00ccc4648203e8c6828bb08203e8 Balance=1000, MntBal=00c5c4648203e8 00ccc4648203e8c6828bb08203e8 Yes

Decode -> Update Nonce -> Encode

Updating the nonce does not change Balance or MntBalances.

Account values before decode Input TokenBal State after DecodeRLP TokenBal after EncodeRLP SlimAccountRLP pyquarkchain TokenBal Aligned
QKC=0, MNT=1000 00c5c4648203e8 Balance=0, MntBalances={100:1000} 00c5c4648203e8 Balance=0, MntBal=00c5c4648203e8 00c5c4648203e8 Yes
QKC=1000, MNT absent 00c7c6828bb08203e8 Balance=1000, MntBalances=empty map 00c7c6828bb08203e8 Balance=1000, MntBal=00c0 00c7c6828bb08203e8 Yes
QKC=0, MNT absent empty Balance=0, MntBalances=nil empty Balance=0, MntBal=nil empty Yes

Decode -> Update Balance -> Encode

When the QKC balance changes and MntBalances == nil, the update logic changes MntBalances to a non-nil empty map to preserve QKC token-entry presence.

Balance update Input TokenBal State after DecodeRLP State after update TokenBal after EncodeRLP SlimAccountRLP pyquarkchain TokenBal Aligned
QKC 1000 -> 0, MNT absent 00c7c6828bb08203e8 Balance=1000, MntBalances=empty map Balance=0, MntBalances=empty map 00c0 Balance=0, MntBal=00c0 00c0 Yes
QKC=0, MNT 1000 -> 0 00c5c4648203e8 Balance=0, MntBalances={100:1000} Balance=0, MntBalances={100:0} (blank) 00c0 Balance=0, MntBal=00c0 00c0 Yes
QKC 0 -> 1000, MNT absent empty Balance=0, MntBalances=nil Balance=1000, MntBalances=empty map 00c7c6828bb08203e8 Balance=1000, MntBal=00c0 00c7c6828bb08203e8 Yes
QKC=0, MNT absent -> 1000 empty Balance=0, MntBalances=nil Balance=0, MntBalances={100:1000} 00c5c4648203e8 Balance=0, MntBal=00c5c4648203e8 00c5c4648203e8 Yes

Conclusion

DecodeRLP splits pyquarkchain's unified TokenBal into Balance for QKC and MntBalances for non-QKC tokens. EncodeRLP merges them back into the same canonical TokenBal bytes.

SlimAccountRLP stores QKC in Balance and MNT in MntBal. A non-nil empty MntBalances is encoded as MntBal=00c0 so it survives the snapshot round trip.

The final EncodeRLP output matches the payload produced by pyquarkchain for all balance states above.

@syntrust

Copy link
Copy Markdown

Checked every form in the tables against 2d589f694 and they hold up: empty, 00c7c6828bb08203e8, 00c6c5648203e8 and the MNT+QKC form all decode and re-encode to identical bytes. No disagreement on any row you listed.

There is one input form the tables don't cover, and it's the one that breaks: TokenBal = 00c0 as a decode input.

00c0 appears in the third table only as an output (the QKC 1000 -> 0 and MNT 1000 -> 0 rows). Those bytes get committed to the trie, so on the next block that touches the account they are what DecodeRLP is handed. The tables implicitly follow one in-memory state through decode → update → encode; the missing step is the state going to disk and coming back.

Filling that row in:

Input TokenBal State after DecodeRLP TokenBal after EncodeRLP SlimAccountRLP pyquarkchain TokenBal Aligned
00c0 Balance=0, MntBalances=nil empty Balance=0, MntBal=nil empty Yes

For the state-transition path that Yes is correct and should stay: pyquarkchain rebuilds _balances from the pair list, which never carries zeros, so a reloaded account starts with an empty map and re-commits as b"" too. Reproducing 00c0 on read-back would diverge in the other direction.

The problem is the SlimAccountRLP column, which is not a state transition. Three paths use it as a pure format conversion and need full -> slim -> full to be the identity:

  • triedb/pathdb/generate.go:749generateRange(..., types.FullAccountRLP); proveRange rebuilds a StackTrie and compares gotRoot != root (same file, 347-353). A 00c0 account re-expands as 0x80, so its range never proves and is regenerated forever.
  • eth/protocols/snap/protocol.go:116AccountRangePacket.Unpack expands peer slim bodies before the range proof is checked against the state root.
  • core/state/stateupdate.go:220,263 stores slim accountOrigin blobs that triedb/pathdb/execute.go:96 replays to reverse state history.

Measured on 2d589f694:

leaf in : f84c01 8200c0 a056e8… 8400000001 80
slim    : c50180808001                          <- MntBal field is gone entirely
leaf out: f84a01 80     a056e8… 8400000001 80

So A non-nil empty MntBalances is encoded as MntBal=00c0 so it survives the snapshot round trip only holds for accounts decoded from a leaf that carried a QKC pair or MNT entries. An account decoded from a 00c0 leaf never reaches that branch, because DecodeRLP leaves MntBalances nil — which also makes the slim.MntBal = []byte{0x00, 0xc0} marker unreachable from any trie-leaf-decode path.

Suggested split: keep the codec lossless (StateAccount faithfully represents a leaf) and move the "reload clears the map" semantics to core/state, which drops the marker when loading an account into a live state object and sets it on every balance write, including writes to zero.

Two TokenBal values in the tables look off by one byte in the MNT pair's inner list header — measured against TokenBalances.SerializeToBytes:

MNT{100:1000}          = 00c5c4648203e8                 (table: 00c6c5648203e8)
MNT+QKC, both 1000     = 00ccc4648203e8c6828bb08203e8   (table: 00ccc5648203e8c6828bb08203e8)
QKC{35760:1000}        = 00c7c6828bb08203e8             (matches)

The pair is c4 + 64 + 8203e8, so the MNT-only payload is 00c5c4648203e8. The QKC-only value is correct and matches the existing pyqkcVecNonce1QKC1000 golden.

@qzhodl

qzhodl commented Aug 13, 2026

Copy link
Copy Markdown

@ping-ke Small correction: the MNT pair hex in the table looks off by one byte. For tokenID=100, balance=1000, the pair is c4 64 82 03 e8, so:

  • QKC=0, MNT=1000: 00c5c4648203e8, not 00c6c5648203e8
  • QKC=1000, MNT=1000: 00ccc4648203e8c6828bb08203e8, not 00ccc5648203e8c6828bb08203e8

The QKC-only value looks correct.

@qzhodl

qzhodl commented Aug 13, 2026

Copy link
Copy Markdown

pyquarkchain has two distinct QKC=0 states: an absent token table ({} -> empty TokenBal) and an explicit zero QKC entry ({QKC:0} -> 00c0). So not every zero QKC balance should encode as 00c0; only the explicit zero-entry case should.

After splitting pyquarkchain's unified token table into Balance plus non-QKC MntBalances, goshard has no explicit place to store that distinction. As a result, MntBalances != nil is now implicitly used as the token-presence marker.

That feels fragile: if any caller leaves MntBalances nil while Balance == 0 is meant to represent {QKC:0}, mergeQKCTokenBalances emits empty bytes instead of 00c0. Please make this marker explicit, or document/test this invariant across state updates, snapshot, and pathdb.

@ping-ke

ping-ke commented Aug 14, 2026

Copy link
Copy Markdown
Author

pyquarkchain has two distinct QKC=0 states: an absent token table ({} -> empty TokenBal) and an explicit zero QKC entry ({QKC:0} -> 00c0). So not every zero QKC balance should encode as 00c0; only the explicit zero-entry case should.

After splitting pyquarkchain's unified token table into Balance plus non-QKC MntBalances, goshard has no explicit place to store that distinction. As a result, MntBalances != nil is now implicitly used as the token-presence marker.

That feels fragile: if any caller leaves MntBalances nil while Balance == 0 is meant to represent {QKC:0}, mergeQKCTokenBalances emits empty bytes instead of 00c0. Please make this marker explicit, or document/test this invariant across state updates, snapshot, and pathdb.

Resolved across the stacked branches.

feature/mnt-core-types (ac492153d) adds an explicit balanceUpdateCount marker to StateAccount. A zero QKC balance is encoded as 00c0 only when this marker is set; a non-nil empty MntBalances is no longer used as the QKC presence marker.

feature/mnt-state (3357d5bdc) updates the marker on actual QKC balance changes and removes the corresponding update on journal revert. Slim account round trips restore the marker explicitly for snapshot/pathdb use.

Consensus DecodeRLP(00c0) still resets the marker and re-encodes as an empty TokenBal, matching pyquarkchain.

Comment thread core/types/state_account.go Outdated
@ping-ke
ping-ke requested a review from qzhodl August 19, 2026 06:13
Comment thread core/types/state_account.go Outdated
@qzhodl

qzhodl commented Aug 27, 2026

Copy link
Copy Markdown

One remaining snapshot-scope concern: if snap sync and pathdb are intentionally out of scope for this PR, that sounds fine, but snapshot database reads still seem in scope.

flatReader.Account currently rebuilds StateAccount manually from SlimAccount and copies only Nonce, Balance, Root, and CodeHash; it does not restore FullShardKey or decode MntBal back into MntBalances. Since MPTDatabase puts the flat reader before the trie reader, a snapshot cache hit can return a StateAccount missing QKC-specific fields, and a later re-commit can write corrupted account bytes back to the trie.

If this fix is intentionally deferred to the next PR, could you please make that explicit in the PR description? That would make the supported snapshot scope of this PR clear.

@syntrust

syntrust commented Sep 1, 2026

Copy link
Copy Markdown

Even if snap sync and pathdb are out of scope, the snapshot database still cannot round-trip a 00c0 leaf: SlimAccountRLP stores the marker, but FullAccountRLP drops it during proof verification and trie regeneration. Please preserve the marker for snapshot leaf reconstruction while keeping normal snapshot reads normalized, or explicitly declare these snapshot paths unsupported as well.

@ping-ke

ping-ke commented Sep 1, 2026

Copy link
Copy Markdown
Author

One remaining snapshot-scope concern: if snap sync and pathdb are intentionally out of scope for this PR, that sounds fine, but snapshot database reads still seem in scope.

flatReader.Account currently rebuilds StateAccount manually from SlimAccount and copies only Nonce, Balance, Root, and CodeHash; it does not restore FullShardKey or decode MntBal back into MntBalances. Since MPTDatabase puts the flat reader before the trie reader, a snapshot cache hit can return a StateAccount missing QKC-specific fields, and a later re-commit can write corrupted account bytes back to the trie.

If this fix is intentionally deferred to the next PR, could you please make that explicit in the PR description? That would make the supported snapshot scope of this PR clear.

The flat-reader integration is intentionally deferred to the next stacked PR (feature/mnt-state), which updates flatReader.Account to restore FullShardKey and decode MntBal into MntBalances.

This PR only introduces the QKC account codec and the extended slim-account representation. It does not by itself claim support for runtime snapshot database reads, pathdb, or snap sync.

I've added a "PRs and Scope" section to the PR description to clarify the scope of this PR and the responsibilities of the following stacked PRs.

// balanceUpdated keeps a changed zero QKC balance encoded as 00c0. It remains
// set after a revert because pyquarkchain restores the previous value by
// writing it back, preserving the zero-valued token entry.
balanceUpdated bool

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As we discussed before, I still think this approach is more complicated and more error-prone than simply using a balance dictionary that includes both MNT and native balances.

@ping-ke
ping-ke marked this pull request as draft September 6, 2026 08:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants